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1.0  Brief  Review  of  the  Language  Measure 


Let  the  plant  behavior  be  modeled  as  a  DFSA  G,  =  (Q,'L,S,ql,Qm)  where  O  is  the  finite  set  of 
states  with  \Q\  =  n  excluding  the  dump  state  ([RW87]  and  [KG95(1)]),  if  any,  and  q,  e  Q  is  the 

initial  state;  2  is  the  (finite)  alphabet  of  events;  2  *  is  the  set  of  all  finite-length  strings  of  events 
including  the  empty  string  e;  the  (total)  function  of  8:Qxi.->Q  represents  state  transitions  and 
S*:  Qxi,*  ->  q  is  an  extension  of  8 ;  and  Qm  c  Q  is  the  set  of  marked  states. 

Definition  1:  A  DFSA  G;- ,  initialized  at  qt  e  Q ,  generates  language  U G, )  =  {?  e  2* :  8  *  ( qt ,s)&  Q\ 
and  marked  sublanguage  Lm ( G, )  =  {s  <=T,*:S*(qi,s)e  Qm } . 

The  language  L(G;)  is  partitioned  into  the  non-marked  language  L° ( G, )  =  L( G, )-Lm( G, )  and 
the  marked  language  Z,„(G,- ) ,  consisting  of  event  strings  that,  starting  from  the  initial  state  qt  e  Q , 
tenninate  at  one  of  the  non-marked  states  in  Q-  Qm  and  one  of  the  marked  states  in  Qm , 
respectively.  The  set  Qm  of  marked  states  is  partitioned  into  0,'n  and  Q~  ,  where  Q,'n  contains  all 
good  marked  states  that  we  desire  to  reach  and  Qm  contains  all  bad  marked  states  that  we  want  to 
avoid,  although  it  may  not  always  be  possible  to  avoid  the  bad  states  while  attempting  to  reach 
the  good  states.  The  marked  language  Lm(Gj)  can  be  further  partitioned  into  L)„(Gr)  and  (G, ) 
consisting  of  good  and  bad  strings  that,  starting  from  the  initial  state  q, ,  terminate  to  Q  'm  and  Qm  , 
respectively. 

Now  we  construct  a  signed  real  measure  // :  21'  sJt  =  (-00,00)  for  quantitative  evaluation  of 
every  event  string  se2*  based  on  state-based  decomposition  of  L(G()  into  null  (i.e.,Z°(G;)), 
positive  (i.e.,  L^(Gr) ),  and  negative  (i.e.,  Um (G,) )  sublanguages  of  L(Gy) . 

Definition  2\  The  language  of  all  strings  that  start  at  state  qt  e  Q ,  and  tenninate  at  state  qj  <=Q ,  is 
denoted  L(qj,qt)  .  Thus,  L(qj,qi)={seL(Gj):S*(qj,s)  =  qj}. 

Definition  3:  The  characteristic  function  that  assigns  a  signed  real  weight  to  state-partitioned 
sublanguages  is  defined  as:  z-{L{p,q)- l]  such  that 


[-1,0)  if  p  g  Qm 


{0}  if  p<£  Qm 
(0,1]  if  p  g  Qm 


independent  of  qj 


Definition  4\  The  event  cost  is  conditioned  on  the  DFSA  state  at  which  the  event  is  generated, 
and  is  defined  as  ir  :2*x£>->[0,1)  such  that  VqjeQ,  Vo>  si,  Vse2*, 

•  n [o’* |  dj ]  =  xjk  e  [0, l)  ;  Tkfjk<\\ 

•  £[0*1^-]  =  0  if  S(qrcrk)  is  undefined; 

•  =  1 ; 

*  a  [O’*  qj  ]  =  n  [ak  I  qj  ]  n  [si  S(qj ,  crk )] . 
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Now  we  define  the  measure  of  any  sublanguage  of  the  l(g,  )  in  terms  of  the  signed  the 
characteristic  function  %  and  the  non-negative  event  cost  re  . 


Definition  5:  The  signed  real  measure  ^  of  a  singleton  string  set  {.v  i  /- f  c/  y ,  )  c  L((f)  g  2s* 
MW)  =  X(<lj)n(s  |  qf  Vs  g  L(qj,qi) . 

The  signed  real  measure  of  L(qj,qi)  is  defined  as: 

(  \ 


/u{L{qj,qi))= 


2>(W) 

S£L(q  j  ,qt) 


is: 


The  signed  real  measure  of  a  DFSA  G, ,  initialized  at  the  state  is  defined  as: 
Hi  =  MUG,))  =  Xy 


Definition  6:  The  state  transition  cost  of  the  DFSA  is  defined  as  a  function  n\Q^Q^[  0,1)  such 
that  Vg,-,  qk&Q,  x(qk\q:)  =  X  ir(cr|<7,-)=  K  ik  and  =0  if  {<J  e  S  :8{q  ,-,cr)}  =  0  .  The 

<TGl.:S(qj,cr)=qk 

n  x  n  state  transition  cost  matrix,  denoted  as  n  -matrix,  is  defined  as: 

^11  ^12  "•  n\ n 

n=  ^21  K11  "•  Kln 

JZn\  Jt n2  ' 


Proposition  1:  Given  a  state  transition  cost  matrix  n  g  9f,x" ,  the  operator  [/-Ft]  is  invertible  and 
the  bounded  linear  operator  [/  -  I  T  |  1  >  0  where  the  matrix  inequality  is  implied  elementwise. 

Proof:  It  follows  from  Definitions  4  and  6  that: 

InL  =  max  X  jXij  =  1  -0  where  6  g  (0, 1) 
i 

Then,  [7  —  TI]  1  is  invertible  and  is  a  bounded  linear  operator  with  norm  ||  [7  —  FE]_1  <  6>_1  [NS82, 

p.  431], 

00  , 

Using  Taylor  series  expansion,  [/-II]  =  X  n  .  Each  element  of  n  is  non-negative,  so  each 

k= 0 

element  of  is  also.  Thus,  [7  —  n]— 1  >  0  elementwise.  ■ 

Wang  and  Ray  [WR02]  and  Ray  and  Phoha  [RP02]  have  shown  that  the  measure 
jUj  =  /./(L(G,  »  of  the  language  Z(G,)  can  be  expressed  as:  ju,  =  Xy^,y  Hj  +  Xi  where  Xi  =  x(<h)  • 

Equivalently,  in  vector  notation:  //  =  1 1  //  +  X  where  the  measure  vector  //  =  [//,  //2  |7  and  the 

characteristic  vector  X  =  [/!/2  ■■■  Xnf  ■  By  Proposition  1,  the  measure  vector  //  is  uniquely 

determined  as:  fu  =  \I  -  IT]-1  X . 
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2.0  Optimal  Control  Using  the  Language  Measure 

We  now  present  the  theoretical  foundations  of  the  unconstrained  optimal  control  of  discrete 
event  systems  [SL98].  Let  S  =  {S°,Sl,---,SN}  be  the  finite  set  of  all  supervisory  control  policies 
for  the  open  loop  plant  automaton  G  where  5°  is  the  null  controller  (i.e.,  no  disabled  events), 
i.e.  L(S°  /  G)  =  L{G) .  Therefore  the  controller  cost  matrix  rtfs'0)  =  n°  =  uplant  is  the  n -matrix  of 
the  open  loop  plant  automaton  G .  For  a  supervisor  Sl,i<={l,2,---,N} ,  the  control  policy 
selectively  disables  certain  controllable  events,  and  therefore  the  following  (elementwise) 
inequality  holds:  rf  =  Tl(Sk )  <  n°  and  L(Sk  /  G)  c  L(G)  \/Sk  e  S  . 

Definition  7:  For  any  supervisor  .S  e  S  and  any  measure  vector  v  e  5?” ,  the  affine  operator 
7(5) :  9t"  ->  5?”  is  defined  as:  T(S)  v  =  n (S)  v  +  X 

Proposition  2\  VS  <=S ,  T(S)  is  a  contraction  operator,  and  there  exists  a  unique  measure  vector 
jLi(S)  such  that  ju(S)  =  T(S)  ju(S ) . 

Proof:  Since  0<rt(5)<n°  elementwise,  and  n°  is  a  contraction  operator,  nfS)  is  also  a 
contraction  operator.  Since  X  is  a  constant  vector,  T(S)  is  also  a  contraction.  Hence,  3  a 
unique  fixed  point  ju(S)  =  T(S)  ju(S) .  • 

Corollary  1  to  Proposition  2:  The  fixed  point  of  the  contraction  operator  T(Sk )  is: 
juk  =[/-nAf1X  where  nk  =  /u(Sk )  and  rf  =n(Sk) . 

Proof:  The  unique  fixed  point  ju(Sk)o f  T(Sk)  satisfies  the  identity  /y(5i’)  =  n(5i’)/y(5i')  +  X .  As 
0  < < nf^0)  elementwise,  we  have  |n(5i)|  <||n(5,0)||o0<l.  Hence,  the  operator  [7  —  n(5,A )]  1 
is  bounded.  ■ 

Corollary  2  to  Proposition  2:  The  operator  [/-nA]  has  a  real  positive  determinant,  i.e., 
Det[I-  rf]>0. 

Proof:  Eigenvalues  of  the  real  matrix  rf  are  located  within  the  unit  circle  and  they  appear  as 
real  or  complex  conjugates.  Therefore,  eigenvalues  of  [/-rf]  have  positive  real  parts.  So, 
Det\I  -  if  ]  is  real  positive.  ■ 

Proposition  3:  Let  >uk  =  min  Me  ■  If  m)  -  0  then  X ,  -  0  and  if  ///  <  0 ,  then  j.  <  0 . 

Proof:  The  DFSA  satisfies  the  identity  juk  =  X  ^um)  +  X,  that  leads  to  the  inequality 

!e{l,  2,  ■•■,«} 

Mj  >(Z,Xjt)Mj  +Xj  =>  0  -  'Ln j^j/j  > Xj •  The  proof  follows  from  (I -SAA > 0 (see  Defs.  3,  4). 

ee  e 

■ 

Corollary  1  to  Proposition  3:  Let  m /  =  max  Me  •  If  /A  then  % t  >0  and  if  m)  >0,  then 

/e{l,2  ,-,n] 

Xj>  o  • 

Proof:  The  proof  is  similar  to  that  of  Proposition  3.  ■ 


3 


Proposition  4:  Given  FU.S'0)  =  11°  =  n plant  and  juk  =  [I  —  n*]  1 X  ,  let  Ylk  1 1  be  generated  from  I  [k  for 

•  4  if  >  0 

k>  0  as  follows:  Vi,  ye  {1,2,  ijth  element  of  nA+1  is  modified  as:  /r,A+1-  =  tt|  if  p)  =0  and 

<4  if  p)  <0 

Ylk  <  n 0  V/c  .  Then,  pk+1  >  pk  elementwise  and  equality  holds  if  and  only  if  nA+1  =  n/f. 

Proof:  pk+x-pk  =  ([/-nA+1]“1-[/-nA+1]“1  )x 
=[/-nA+1]_1([/-nA]-[/-nA+1])[/-nA] 'x 
=[i-nk+1]  (nA+1  -nk)jik 

Defining  the  matrix  Ak  =  Uk+]  -  nA ,  let  the  j'h  column  of  Ak  be  denoted  as  Aa  .  Then,  Aa  <  0  if 
/jk  <0  and  A kj  >0  if  pk>  0,  and  the  remaining  columns  of  Ak  are  zero  vectors.  This  implies: 
Akjuk  =  2 A*  juk  >0.  Since  nA<n°V£,  [/-n*+lr'>o  elementwise,  we 

j 

have.  [I-nk+1]~lAkpk  >0=>pk+1  >  juk .  For  pk  *  0  and  Ak  as  defined  above,  Akpk  =0  if  and  only 
if  Ak  =  0 .  Then,  nA+1  =  Uk  and  juk+l  =  / .  m 

Corollary  1  to  Proposition  4:  Let  pk  <  0.  Let  nA+1  be  generated  from  nA  by  disabling 
controllable  events  that  lead  to  the  state  .  Then,  //j 4 1  <  o . 

Proof:  Since  only  jth  column  of  [/-nA+1]  differs  from  that  of  [ /  —  1 1 A  |  and  the  remaining 
columns  are  the  same,  the  jth  row  of  the  cofactor  matrix  of  [/-nA+1]  is  the  same  as  that  of  the 
cofactor  matrix  of  \I  -  rf  ] ,  we  have  T>efi/-nA+1]/uA+1  =  Det\l-Wk\pk .  By  Corollary  2  to 
Proposition  2,  both  determinants  are  real  positive.  ■ 

Remark  1:  In  Proposition  4,  some  elements  of  the  jth  column  of  nA  are  decreased  (or 
increased)  by  disabling  (or  re-enabling)  controllable  events  that  lead  to  states  qj  for  which 

/jk  <0  (or  /./A  >0  ).  ■ 

Proposition  5:  Iteration  of  the  algorithm  in  Proposition  4  leads  to  an  optimal  cost  matrix  n*  that 
maximizes  performance  vector  p  =  [/  —  II]  1  X  elementwise. 

Proof:  Let  there  be  another  cost  matrix  n<n°  for  which//  =  [/-nL1  X .  We  will  show  that 
ju<p*.  Starting  with  ju-  ju*  =  [I- fl]  1  [fi  - n*]  p  * ,  we  rearrange  the  elements  of  the  p  *  -vector 
such  that  p  =[ p*  ■■•//,*  u*iA  ■  ■  ■  p*  ]T  where  pk  >0  and  no  controllable  event  leading  to  states 

v - v - — '  ^ — - v - ' 

>0  <0 

qk  has  been  disabled;  and  pk<  0  for  k  =  t  +  \,t  +  2,---,n  where  all  controllable  events  leading  to 
states  qk ,  k  =  \,  ,  have  been  disabled.  The  cost  matrices  ft  and  n*are  also  rearranged  by 

columns  in  the  order  in  which  the  p*  -vector  is  arranged. 
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The  algorithm  in  Proposition  4  keeps  elements  in  the  first  t  columns  of  n*  the  same  as  those 
of  the  (open  loop  plant’s)  n°  -matrix  and  decreases  the  elements  in  the  last  (n  - 1)  columns  to  the 
maximum  pennissible  extent  by  disabling  all  controllable  events.  In  contrast,  the  columns  of  ft 
are  reduced  by  an  arbitrary  choice.  Therefore,  the  (ft  -11*)  -matrix,  whose  first  t  columns  are 
non-positive  and  last  (/?-/)  columns  are  non-negative,  yields: 

ju-ju*  =  [/-II]_1  [first  t  cols  <0  \  last  (n  -€)  cols  >  0\  ju*  where  ju*  =  [ju*  ■  ••/./)  /j*+]  •■■/./„*  ]r  . 

>0  <0 


Since  [/-ft]  1  >0  elementwise,  we  conclude  that 


/?-//*=  [T  -  ft]  1 


>0 


ICo/-//  +  E  CoL-ju* 
j=(+ 1 


<0 


<0  <0 

Therefore,  fi  <  ju*  for  any  choice  of  ft  . 


Proposition  6:  The  control  policy  induced  by  the  n* -matrix  is  unique  in  the  sense  that  the 
controlled  language  is  most  permissive  (i.e.,  least  restrictive)  among  all  controller(s)  having  the 
best  performance. 


Proof:  Disabling  controllable  event(s)  leading  to  a  state  qj  with  performance  measure  /j*  =  0 
does  not  alter  the  perfonnance  vector  ju* .  The  optimal  control  does  not  disable  any  controllable 
event  leading  to  a  state  with  zero  performance.  Thus,  the  control  policy  induced  by  the  n‘- 
matrix  is  most  permissive,  among  all  controllers  with  equal  performance  /u* .  m 

Remark  2:  Propositions  5  and  6  suffice  to  conclude  that  the  n* -matrix  yields  the  most 
pennissive  controller  with  the  best  performance  n*  .  The  control  policy  is  realized  as  follows: 

•  All  controllable  events  leading  to  the  states  qj  where  ju*  <  0  are  disabled; 

•  All  controllable  events  leading  to  the  states  qj  where  ju*  > 0  are  enabled.  ■ 


3.0  Experimentation 

The  experimentation  for  our  research  was  designed  to  resolve  the  issue  of  detennining  the 
n  values  of  a  given  plant  model  for  a  system.  It  is  important  to  be  able  to  mathematically  or 
experimentally  determine  these  values,  as  opposed  to  their  being  set  by  hand,  because  in  general 
it  is  impossible  to  properly  assign  by  hand  probabilities  for  uncontrollable  events  (it  is  possible 
to  assign  the  n  values  for  controllable  events,  since  a  controller  can  choose  how  often  they  wish 
such  events  to  occur).  For  typical  MICA  scenarios,  we  used  the  Boeing  Simulator  to  generate 
the  necessary  data  to  detennine  the  n  values. 

The  simulation  runs  generates  situations  where  events  occur.  Specifically,  we  implemented  an 
event  generator  that  examined  the  variables  of  the  system  and  detennined,  from  those  variables 
and  the  current  state  of  the  system  according  to  the  plant  model,  what  discrete  event  is  occurring 
in  the  system.  Since  the  Boeing  simulator  is  a  rich  simulation  environment,  we  can  use  it  to 
determine  the  plant  model's  n  values. 


5 


Experiment  Description 


The  basis  for  the  experiment  is  this  hypothesis:  Hypothesis:  Jt  values  can  be  determined 
through  event  generation  and  statistical  analysis  of  those  events  occurring  over  the  course  of  a 
large  number  of  simulated  or  actual  runs. 

The  goal  of  the  experiment  is  to  show  that  the  experimental  data  for  these  if  values  converges  to 
within  s  of  its  actual  value  with  probability  8.  The  experimental  procedure  was  as  follows: 

1 .  Identify  the  states  and  events  (and  thus  the  transition  table)  for  the  system’s  plant  model. 

2.  Implement  event  generators  that  take  the  simulator/experiment  data  and  detennine  if  one 
of  the  previously  defined  events  have  occurred.  Then  update  the  plant  model  to  the  new  state 
designated  by  the  event  that  occurred  and  the  previous  state. 

3.  Tally  event  occurrences  in  a  if -like  matrix.  That  is,  an  entry  in  an  m  *  n  matrix  is 
incremented  whenever  the  corresponding  event  occurs  at  the  corresponding  state. 

4.  Sum  the  values  of  a  row  i  and  then  divide  each  element  in  i  by  that  sum.  Scale  this 
number  by  (1-9)  for  some  pre-determined  value  of  9.  The  resulting  matrix  contains  the  if 
values. 

5.  Repeatedly  run  experiment  to  continue  adding  to  the  event  occurrence  matrix.  Over  the 
course  of  numerous  experiment  runs,  the  resulting  if  value  matrix  from  these  runs  should 
converge. 

In  order  to  conduct  the  experiment,  we  implemented  event  generators  that  would  determine,  at 
each  tick,  whether  or  not  an  event  in  the  set  of  possible  events  given  the  current  state  had 
occurred.  For  example,  a  event  generator  for  a  Damaged  event  would  trigger  if  the  simulator's 
status  for  a  given  platform  indicated  that  the  platform  was  damaged.  Each  event  required  an 
event  generator.  Once  these  generators  were  developed,  we  could  conduct  the  experiment. 


Simulation 

The  experiment  scenario  was  that  of  two  fighters  and  five  targets.  Each  fighter  had  the  same 
plant  model,  and  each  fighter's  events  were  used  in  the  tallying  of  events  in  the  event  occurrence 
matrix.  Targets  were  randomly  placed  on  the  map  and  fighters  emerged  from  a  location 
representing  their  base  on  the  map  (the  southeast  corner).  See  Figure  1. 
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Figure  1.  Sample  Experiment  Scenario 


The  fighters  would  emerge  randomly  from  the  base,  and  upon  detection  of  a  target,  fly  toward  it 
to  engage,  and  then  decide,  based  upon  uncontrollable  events  and  preset  controllable  event 
probabilities,  what  actions  to  take.  The  fighter's  processes  continued  in  this  manner  until  they 
were  destroyed,  all  the  targets  were  destroyed,  or  an  upper  limit  on  time  for  each  run  was 
reached. 

Plant  Model 

The  experiment  plant  model  had  these  states: 

1 .  AbortMission  -  fighter  has  stopped  attempting  combat/flight  toward  combat 

2.  AtBase  -  fighter  is  at  base 

3.  Attack  -  Fighter  has  attacked  or  is  attacking  target 

4.  AttackWithDamaged  -  fighter  is  in  combat  and  is  attacking  (again)  with  damages  to  itself 

5.  AttackWithLowFuel  -  fighter  is  in  combat  and  is  attacking  (again)  with  low  fuel 

6.  DamagedBeforeAttack  -  fighter  is  not  in  combat  and  has  received  damage 

7.  DamagedDuring Attack  -  fighter  is  in  combat  and  has  received  damage 

8.  EnRoute  -  fighter  is  flying  toward  target/waypoint 

9.  LowFuelBefore Attack  -  fighter  is  not  in  combat  and  is  nearing  insufficient  fuel  to  return  to 
base 

10.  LowFuelDuringAttack  -  fighter  is  in  combat  and  is  nearing  insufficient  fuel  for  returning  to 
base 

1 1 .  MidairRefuel  -  fighter  is  attempting  to  refuel  from  a  tanker 

12.  NearTg  -  Fighter  is  within  range  of  target 

13.  NoWeapons  -  fighter  has  no  weapons 

14.  SelfDestroyed  -  fighter  is  destroyed 

15.  TargetDestroyed  -  fighter's  current  target  has  been  destroyed 
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The  following  events  were  recognized  by  the  generators,  where  U  stands  for  uncontrollable,  and 
C  for  controllable: 

a.  attack  (C)  -  fighter  decides  to  attack  its  current  target 

b.  damaged  (U)  -  fighter  is  damaged 

c.  die  (U)  -  fighter  has  been  destroyed 

d.  flee  (C)  -  fighter  told  to  flee  to  base 

e.  lowFuel  (U)  -  lighter  near  insufficient  fuel  to  return  to  base,  given  current  position 

f.  midairRefuel  (C)  -  fighter  decides  to  refuel  from  a  tanker 

g.  nearTg  (U)  -  fighter  is  near  its  target 

h.  noWeapons  (U)  -  fighter  has  used  all  available  weapons  (includes  losing  weapons  to 
damage,  jamming,  etc.) 

i.  reachBase  (U)  -  lighter  reaches  base 

j.  refuelCompleted  (U)  -  fighter  has  finished  a  midair  refueling 

k.  replan  (U)  -  fighter  has  to  change  its  course  due  to  uncontrollable  event 

l.  tgDestryd  (U)  -  fighter's  target  has  become  destroyed 

Transition  Table 

The  entries  of  a  transition  table  indicate  the  events  that  are  possible  from  each  state.  For 
example,  from  state  1  (atBase),  only  one  event  is  possible,  replan,  and  that  event  takes  the  model 
to  state  2  (EnRoute).  Table  1  is  the  transition  table  for  the  plant  model. 


Table  1:  Transition  Table  for  Plant  Model 


a 

b 

c 

d 

e 

f 

S 

h 

i 

i 

k 

1 

1 

14 

2 

2 

8 

3 

3 

7 

14 

1 

10 

13 

8 

15 

4 

4 

14 

1 

13 

8 

15 

5 

5 

14 

1 

13 

8 

15 

6 

14 

1 

7 

8 

7 

4 

14 

1 

8 

8 

6 

14 

9 

12 

13 

8 

9 

14 

1 

11 

10 

8 

10 

5 

14 

1 

11 

8 

11 

14 

8 

12 

3 

14 

13 

8 

15 

13 

14 

1 

14 

15 

14 

8 

Results 

After  several  hundred  simulation  runs,  the  entries  with  frequently  occurring  events  began  to 
stabilize  near  a  particular  value.  For  example,  the  last  few  n  values  for  the  transition  <Attack, 
attack,  Attack>  (from  the  Attack  state,  choose  to  attack)  are  shown  just  below  in  Table  2.  The 
resulting  n  matrix,  prior  to  any  scaling,  is  in  Table  3  (figures  may  not  sum  to  1  due  to 
rounding). 
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0.439537 

0.439553 

0.43964 

0.439727 

0.439835 

0.439878 

0.439864 

0.439893 

0.439879 

0.439843 


Table  2:  Value  Stabilization 

Note  that  some  entries  are  0,  despite  their  having  potential  for  having  a  transition  occur  there, 
according  to  the  transition  table.  There  are  a  few  possible  reasons  for  this  result.  One  is  that  not 
enough  runs  might  have  been  conducted  on  the  model  to  get  enough  situations  for  those 
transitions  to  occur.  Another  more  likely  reason,  is  that  the  logic  of  the  fighters  do  not  allow  the 
fighters  to  reach  those  situations,  which  means  that  those  transitions  are  a  part  of  a  inaccurate 
modeling  of  the  system  at  hand.  If  the  fighters,  or  the  plant  model,  were  modified  to  better 
model  the  actual  situation,  such  anomalies  would  be  reduced.  Thus  this  method  establishes 
n  values  for  the  available  transitions,  and  it  also  points  out  transitions  which  may  not  actually 
have  a  counterpart  in  the  real  scenario,  be  it  from  fighters  whose  logic  precludes  those  situations 
from  happening,  or  otherwise.  The  difficulty  in  creating  a  plant  model  for  a  given  system  makes 
such  capability  in  this  method  algorithm  a  bonus.  After  refining  the  fighter  model,  plant  model, 
and  scaling,  we  have  a  usable  n  table,  on  which  we  can  use  the  language  measure. 


a 

b 

C 

d 

e 

f 

g 

h 

i 

i 

k 

1 

1 

.09 

.91 

2 

1 

3 

.44 

.01 

.02 

.1 

.006 

.11 

.01 

.3 

4 

0 

0 

0 

0 

0 

0 

5 

.41 

.05 

.11 

.19 

0 

.24 

6 

0 

0 

.64 

.36 

7 

0 

0 

.69 

.31 

8 

.003 

.003 

.1 

.83 

.02 

.04 

9 

0 

.32 

.67 

.01 

0 

10 

.58 

0 

.16 

.18 

.08 

11 

.27 

.73 

12 

.99 

.006 

.003 

0 

0 

13 

.001 

.99 

14 

15 

0 

1 

Table  3:  n  Matrix 
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Experiment  2:  Robustness  of  measure  and  optimality  algorithm 

The  scenario  developed  for  the  simulation  experiments  in  this  setting  deployed  a  fighter  airplane 
against  a  target  that  attempts  to  defend  itself  by  shooting  down  the  fighter.  The  experiment  was 
conducted  using  four  different  but  similar  plant  models  and  four  increasingly  aggressive 
controllers  [RW87].  A  nominal  plant  model  was  developed  representing  a  single  fighter  aircraft 
on  a  mission  to  destroy  a  single  defensive  target  (e.g.,  an  anti-aircraft  artillery  site),  as  seen 
previously  in  prior  presentations.  From  this  plant,  3  other  models  were  created  through  small 
variations  in  the  n  values  of  transitions  and  X  values  of  the  marked  states.  These  variations 
allow  the  experiment  to  examine  the  language  measure's  validity,  and  utilize  the  X  vector  feature 
of  assessing  the  relative  worth  of  marked  states.  The  four  models  of  the  plant  aircraft  are 
described  below: 

Plant  model  1 :  The  nominal  plant  model,  where  the  values  of  the  n  matrix  were  assigned  by 
their  likelihood  of  occurrence  based  on  limited  available  data.  The  X  values  were  set  to  equal 
magnitude  weights  for  the  two  main  outcomes  of  fighter  destruction  and  target  destruction. 
Specifically,  X  values  assigned  to  these  states:  aborting  the  mission  (-.05),  target  destruction  (1), 
and  fighter  destruction  (-1.0).  All  other  states,  being  unmarked,  have  a  X  value  of  zero. 

Plant  model  2:  Same  X  values  as  in  Plant  model  1,  but  the  lvalues  corresponding  to  the 
controllable  events  at  each  state  are  equal  and  their  sum  unchanged.  The  remaining  n  values 
are  not  altered. 

Plant  model  3:  Same  k  values  as  those  in  Plant  model  1,  but  different  X  values  to  represent  the 
preference  of  preserving  the  fighter  over  target  destruction.  Specifically,  target  destruction  has  a 
X  value  of  .95,  while  fighter  destruction  has  a  X  value  of  -1.  The  remaining  X  values  are  the 
same  as  those  in  Plant  model  1 . 

Plant  model  4:  Same  n  values  as  in  Plant  model  2,  and  X  values  as  in  Plant  model  3. 

The  differences  between  the  values  of  the  various  plant  models  are  small,  between  0.01  to  0.2. 
The  states  and  events  used  to  represent  the  fighter-target  scenario  are  similar  to  the  ones  from  the 
n  value  generation  experiment. 

Four  supervisors  were  designed  from  four  simple  specification  sets,  which  are  described  below 
in  order  of  increasing  aggressiveness: 

Specification  set  1:  Only  attack  if  there  are  no  problems  with  the  fighter.  That  is,  no  damage 
(major  or  minor),  and  ample  fuel.  Fighter  shall  not  start  attacking  if  there  are  any  problems,  and 
if  a  problem  (including  running  low  on  fuel)  develops  during  attack,  the  mission  will  be  aborted. 
Mid-air  refueling  is  not  allowed  during  attack. 

Specification  set  2:  Attack  at  least  once,  but  then  abort  if  damaged  at  all.  Continue  to  attack 
otherwise.  If  the  fighter  already  has  damage,  it  will  attack  once  and  then  abort.  If  the  fighter  is 
undamaged  when  attack  begins,  it  shall  abort  once  damage  has  taken  place.  No  specifications 
with  respect  to  fuel  levels. 
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Specification  set  3:  Abort  the  attack  if  the  fighter  has  major  damage,  and  attack  at  most  twice 
with  minor  damage,.  Otherwise  attack  until  the  target  is  destroyed  or  weapons  have  run  out.  (A 
new  state  is  added  to  handle  counting  in  the  case  of  minor  damage  incurred.)  If  the  fighter 
already  has  major  damage  before  attacking,  it  will  abort  and  not  attack.  If  the  fighter  incurs 
major  damage  while  attacking,  the  mission  shall  be  aborted.  If  the  fighter  has  minor  damage 
prior  to  attacking,  it  will  attack  at  most  twice  and  then  abort.  If  the  fighter  incurs  minor  damage 
during  attack,  it  will  attack  once  more,  then  abort.  No  restrictions  based  on  fuel  level. 

Specification  set  4:  Attack  regardless  of  problems  until  target  is  destroyed  or  all  appropriate 
weapons  have  been  used.  Abort  if  all  available  weapons  have  been  used. 

Supervisors  were  designed  for  the  open-loop  plant  models  based  on  the  above  four  specifications 
and  were  labeled  according  to  their  specification  set  number. 

For  controllers  1,  2,  and  4,  supervisor  construction  and  application  reduces  to  simple  removal  of 
states  and  transitions  from  the  open-loop  plant  model.  For  controller  3,  an  additional  state  was 
added  to  handle  the  counting  required  when  considering  what  the  fighter  should  do  when  it  has 
incurred  minor  damage.  If  it  is  not  obvious  how  to  add/delete  states  and  events  to/from  the  open 
plant  model  when  applying  a  controller,  product  construction  is  used  and  the  language  measure 
is  then  applied  to  that  construction. 

Results 

Language  measurements  were  carried  out  using  each  of  the  four  controllers  on  each  of  the  four 
plants,  as  well  as  the  unsupervised  version  of  the  plants,  for  a  total  of  20  measurements.  Table  4 
lists  the  results  of  these  measurements.  The  No  Controller  row  shows  the  language  measure  of 
the  open  plant  itself,  without  a  controller  being  applied  to  it.  Those  figures  are  the  baseline  for 
comparison  of  controllers  that  are  applied  to  those  models.  Controllers  that  measure  less  than  the 
open  plant  model's  measure  are  considered  to  be  bad  or  useless  controllers  (and  thus  their 
specifications  detrimental):  it  would  be  better  to  attempt  the  scenario  without  those  controls 
being  applied,  for  there  is  a  better  chance  of  success  according  to  the  measure. 


Table  4:  Results  of  the  20  measurements 


Open  Plant  Models 

No  Control 

.09898 

.1328 

.07544 

.106 

Controller  1 

.0797 

.104 

.0667 

.08972 

Controller  2 

.07698 

.09547 

.06425 

.08054 

Controller  3 

.10311 

.13435 

.08422 

.1131 

Controller  4 

.09424 

.1175 

.08167 

.1034 

According  to  the  table,  controller  3  was  the  only  controller  that  managed  to  improve  upon  all 
four  of  the  open-loop  plant  models.  As  such  it  consistently  was  the  best  control  specifications  to 
use  for  the  scenario  of  the  four  that  were  proposed.  The  consistency  is  important,  because  the 
measure's  validity  depends  on  being  consistent  even  over  small  perturbations  of  plant  models 
like  the  changes  that  were  in  this  experiment.  Furthermore,  the  order  of  goodness  according  to 
the  measure  was  also  consistent  across  the  four  varieties  of  plant  models:  controller  3  was  best, 
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followed  by  2,  1,  and  lastly  4.  This  observation  further  establishes  the  consistency  of  the 
measure.  If  the  ranking  of  controllers  could  be  permuted  when  they  differed  by  non-trivial 
amounts  as  the  result  of  small  perturbations  in  the  plant  model,  the  measure  would  lose  validity. 

Controllers  1  and  4  were  both  bad  strategies,  they  consistently  yielded  measures  worse  than  that 
of  the  unrestricted  plant,  while  the  performance  of  controller  2  was  mixed;  its  measures  were  so 
close  to  those  of  the  original  plant  that  its  measure  compared  to  the  original  plant  was  very 
sensitive  to  the  changes  of  the  X  vector  and  Tl-matrix.  The  all-or-nothing  strategies  failed  to 
rank  as  highly  as  the  strategies  that  were  middling  in  their  aggressiveness,  controller  1  being  too 
risk  averse  and  controller  4  allowing  for  too  much  risk.  The  results  of  the  experiment  suggest 
that  over  the  course  of  repeated  runs  of  this  scenario,  using  the  range  of  parameter  perturbation 
in  this  experiment),  controller  3  would  yield  the  highest  percentage  of  successful  runs,  followed 
by  2,  1  and  4. 

Now  we  compare  these  results  to  those  of  applying  optimal  control  onto  all  of  the  previous 
measurements  in  Table  5. 


Table  5:  Results  from  optimal  algorithm 


w/  Optimal 

Open  Plant  Models 

Open  Plant 

.12298 

.16143 

.10286 

.13618 

Controller  1 

.0797 

.104 

.0667 

.08972 

Controller  2 

.09289 

.11545 

.08132 

.10219 

Controller  3 

.10564 

.13573 

.08844 

.11365 

Controller  4 

.09514 

.11847 

.08257 

.10438 

As  expected,  unconstrained  optimal  control  yields  the  best  results.  Controller  l's  performance  is 
unchanged  after  optimization,  which  means  that  it  was  already  optimal  for  the  set  of 
specifications  it  was  covering.  Controller  2's  performance  has  a  significant  increase,  instead  of 
being  the  worst,  it  is  now  better  than  controller  1 . 
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4.0  Conclusions 


With  the  process  of  determining  n  values  from  the  first  experiment,  we  have  enabled  the 
language  measure  to  be  applicable  on  any  system  that  can  be  represented  as  a  discrete  event 
plant.  The  Boeing  Simulator  was  used  as  a  model  of  the  real  world,  and  events  can  be  extracted 
from  it,  or  other  simulations,  or  even  the  real  world,  through  the  process  detailed  above.  The 
first  experiment  helps  to  shape  the  plant  model  and  also  provides  accurate  %  values  to  use  with 
that  plant  model.  Given  those  results,  one  can  apply  the  language  measure  on  control 
specifications  to  determine  which  specifications  are  best.  Alternatively  one  can  use  the 
optimality  algorithm  to  find  the  best  set  of  enabled  and  disabled  controllable  events  under  the 
given  set  of  specifications  for  the  controller. 
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